Kernels , Margins , and Low - dimensional Mappings ∗
نویسندگان
چکیده
Kernel functions are typically viewed as providing an implicit mapping of points into a high-dimensional space, with the ability to gain much of the power of that space without incurring a high cost if the result is linearly-separable by a large margin γ. However, the Johnson-Lindenstrauss lemma suggests that in the presence of a large margin, a kernel function can also be viewed as a mapping to a low-dimensional space, one of dimension only Õ(1/γ). In this work, we explore the question of whether one can efficiently produce such low-dimensional mappings, using only black-box access to a kernel function. That is, given just a program that computes K(x, y) on inputs x, y of our choosing, can we efficiently construct an explicit (small) set of features that effectively capture the power of the implicit high-dimensional space? We answer this question in the affirmative if our method is also allowed black-box access to the underlying data distribution (i.e., unlabeled examples). We also give a lower bound, showing that if we do not have access to the distribution, then this is not possible for an arbitrary black-box kernel function. Our positive result can be viewed as saying that designing a good kernel function is much like designing a good feature space. Given a kernel, by running it in a black-box manner on random unlabeled examples, we can efficiently generate an explicit set of Õ(1/γ) features, such that if the data was linearly separable with margin γ under the kernel, then it is approximately separable in this new feature space.
منابع مشابه
Complex Transfinite Barycentric Mappings with Similarity Kernels
Transfinite barycentric kernels are the continuous version of traditional barycentric coordinates and are used to define interpolants of values given on a smooth planar contour. When the data is two-dimensional, i.e. the boundary of a planar map, these kernels may be conveniently expressed using complex number algebra, simplifying much of the notation and results. In this paper we develop some ...
متن کاملActions, Norms, Subactions and Kernels of (Fuzzy) Norms
In this paper, we introduce the notion of an action $Y_X$as a generalization of the notion of a module,and the notion of a norm $vt: Y_Xto F$, where $F$ is a field and $vartriangle(xy)vartriangle(y') =$ $ vartriangle(y)vartriangle(xy')$ as well as the notion of fuzzy norm, where $vt: Y_Xto [0, 1]subseteq {bf R}$, with $bf R$ the set of all real numbers. A great many standard mappings on algebr...
متن کاملQuantized Kernel Learning for Feature Matching
Matching local visual features is a crucial problem in computer vision and its accuracy greatly depends on the choice of similarity measure. As it is generally very difficult to design by hand a similarity or a kernel perfectly adapted to the data of interest, learning it automatically with as few assumptions as possible is preferable. However, available techniques for kernel learning suffer fr...
متن کاملFlexible Tree Kernels based on Counting the Number of Tree Mappings
Functions counting the number of common sub-patterns between trees have been promising candidates for kernel functions for trees in learning systems. There are several viewpoints of how two patterns between two trees can be regarded as the same. In the tree edit distance, these viewpoints have been well formalized as the class of tree mappings, and several distance measures have been proposed a...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2004